My Experience with NIST ARIA

In October I competed in an online LLMs security competition as part of the NIST Assessing Risks and Impacts of AI program. It was hosted by Humane Intelligence, which was supported by NIST and CISA. They put out the call for people across the nation to break production LLMs submitted by model creators. Humane Intelligence covered the flights, hotel, and conference costs for the top 30 scoring participants to attend CAMLIS and do a similar exercise working together in person. I was one of the 30 who beat out the competition (there were over 500 participants) and was able to attend the conference in Washington DC (technically Arlington, Virginia but you could see the capitol building from the room we were red teaming in so I call it what I want).

I got a lot out of this opportunity. I was able to be a part of a team from around the country, (including security engineers at Dropbox, Amazon, and Microsoft), which attacked real life software platforms used by fortune 50 companies. The National Institute for Standards and Technology, who sponsored the event, was represented there and will use the data and vulnerabilities we found to inform their next standards paper aiming to advance how organizations secure their generative AI models.

We were literally writing the book on AI safeguards.

The event was sponsored by NIST, and we had people from NIST and CISA (Cybersecurity and Infrastructure Security Agency) there to watch and talk with us, as well as industry leaders in the generative AI space. I found multiple vulnerabilities in productivity and office software from multiple vendors.

The conference itself was also an amazing opportunity, I watched talks from AI security experts from leading institutions such as Microsoft, Google Cloud, Booze Allen and Hamilton, and Meta, letting me learn the bleeding edge of security in AI. The talks that I watched gave me a few ideas of projects and research I want to do at UWEC.

After the conference, we were given a chance to work together and write a paper out of the results form the event, I'll update this once that gets published.